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ABSTRACT 

A study was done of the relationship between the 
American College Testing (ACT) Assessment and the ACT/COMP (College 
Outcomes Measures Program) test and general intellectual ability of 
college students. The subjects for the study were 133 undergraduates, 
mostly freshmen, in Introductory Psychology at the University of 
Tennessee, Knoxville. The subjects took the Multidimensional Aptitude 
Battery (MAB) , a group-administered intelligence test patterned after 
the Wechsler Adult Intelligence Scale. Results of the MAB were 
compared through correlation analysis and five regression analyses 
with the students 1 ACT and ACT/COMP scores as retrieved from 
university records. Variables used included the ACT and ACT/COMP 
scores, the MAB total score, high school grade point average, and 
college grade point average (which was measured 2 years later at the 
end of the sophomore year). Results showed that the students 1 scores 
on the ACT and ACT/COMP exams correlate strongly with intelligence as 
measured by the MAB. The ACT/COMP exam had no relationship with high 
school grade point average. Intelligence was a strong predictor of 
ACT Assessment scores, and achievement, while significant, did not 
explain much of the variance in scores. Overall it appeared that both 
the ACT Assessment and the ACT/COMP are primarily measures of general 
intellectual ability rather than achievement. Included are 13 
references. (JB) 
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JUST WHAT DOES THE ACT ASSESSMENT AND 
ACT/COMP MEASURE ANYWAY 

Abstract 

The ACT Assessment and ACT/COMP are two important tests used 
in colleges and universities , but no published research has 
examined the relationship between these measures and general 
intellectual ability* This study found that students' scores on 
these exams correlate strongly with intelligence as measured by 
the Multidimensional Aptitude Battery. Regression models that 
predicted Sophomore Grade Point Average from intelligence or Act 
Assessment scores and high school GPA explained 98% of the 
variance. Similarly high levels of variance in ACT/COMP scores 
were explained by these same variables * It appears that both the 
ACT Assessment and the ACT/COMP are primarily measures of general 
intellectual ability. 



ERLC 



4 



ACT and Intelligence/3 



JUST WHAT DOES THE ACT ASSESSMENT AND 
ACT/COMP MEASURE ANYWAY 

The ACT Assessment is one of the most commonly used college 
admission tests in the United States, and the ACT/COMP exam is 
the oldest test for college outcomes assessment. Both these 
tests claim to be measures of achievement, but the ACT Assessment 
is sometimes used as a measure of general intellectual ability. 
The purpose of this paper is to examine the relationship between 
the ACT Assessment, the ACT/COMP, and general intellectual 
ability. 

According to the ACT Technical Manual, the ACT Assessment is 
a work sample that measures both students' current level of 
achievement and their ability to perform college level work (ACT, 
1973) . As such, the exam is often used to predict college grades 
(Snowman, Leitner, Snyder, and Lockhart, 1980) and sometimes 
other student qualities such as reading levels (Ashmore and Cork, 
1985) . Various studies have used these test scores as measures 
of general intellectual ability (Pedrini, Tritchler, and Pedrini, 
1984; Savage, 1983), but no published study has ever examined the 
relationship between the ACT Assessment and general intellectual 
ability. 

The ACT/COMP is a test that measures the ability of an 
individual to function effectively as an adult (Forrest and 
Steele, 1982) . As such, this exam is considered by ACT to be a 
measure of college achievement and the company recommends its use 
in college outcomes assessment (Steele, 1989). Some researchers, 
however, feel that the COMP exam measures little other than 
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general verbal intelligence (Pike and Banta, 1989) and, 
therefore , it is not sensitive to college educational influences* 

Considering the growth of assessment in colleges and 
universities and the continuing use of outcomes assessment 
instruments like the ACT/COMP exam, a study of the relationship 
between general intelligence and the ACT/COMP exam is in order. 
Other studies that have examined the relationship of ability and 
the ACT/COMP exam have used the ACT Assessment as the measure of 
ability (Pike and Banta, 1989). This makes sense if the ACT 
Assessment measures general intellectual ability, but if it 
measures high school achievement as the test company claims (ACT, 
1973) then the relationship is one between levels of achievement 
rather than levels of ability. 

No published study to date has examined the relationship 
between intelligence and either the ACT Assessment or the 
ACT/COMP exam. This study is designed to fill that void. 

METHODS 

Subjects 

Subjects for this study were 133 undergraduates in 
Introductory Psychology at the University of Tennessee, Knoxville 
(UTK) . The subjects received two points of fextra credit in their 
class for participating in this study. They were mostly 
freshmen, 18 or 19 years of age, and 64 percent of them were 
females. 
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Instruments 

ACT Assessment 

The ACT Assessment is an exam designed to assess the 
potential of individuals for college. It was developed to 
provide a work sample that would both measure the achievement in 
high school of the college canidate and give an example of the 
canidate's likely college performance (ACT, 1973). Three scores 
are reported from this exam - English, Math, and Total Score. 
The scores can range from 0 to 35 with most students scoring in 
the 15 to 30 range. 

ACT College Outcomes Measures Program Exam 
The ACT College Outcomes Measures Program exam (ACT/COMP) 
was developed for use in the assessment of general education. 
Using a unique format of visual and audio displays combined with 
process type questions, the test developers claim to measure a 
student's level of effective adult functioning (Forest and 
Steele, 1982) . This exam is the oldest of the general education 
assessment tests, and it has been used by many institutions for 
outcomes evaluation. 

The test consists of six subscales that are added to form a 
total score. These six subscales are Functioning within Social 
Institutions, Using Science, Using Arts, Communication, Solving 
Problems, and Clarifying Values. Total scores on this exam range 
from 130 to 240 with an average score for seniors of 185 and a 
standard deviation of 15. 
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The University of Tennessee, Knoxville, as a part of a 
longitudinal study of ACT/COMP scores, administered this exam to 
freshmen each fall from 1983 to 1989. Scores from these freshman 
administrations were available to the investigator and were used 
in this analysis. 

Multidimensional Aptitude Battery 

The Multidimensional Aptitude Battery (MAB) is a group 
administered intelligence test patterned after the Wechsler Adult 
Intelligence Scale (Jackson, 1984) . This exam consists of two 
subscales of five parts, each of which are summed to provide 
verbal and performance intelligence scores. These two subscales 
are added to form a full scale measure of intelligence. The test 
is a paper and pencil exam and can be administered in groups of 
any size. 

Scores can be obtained either on a traditional I.Q. scale 
(mean of 100 and standard deviation of 10) or a standard scale 
such as that used with the Scholastic Aptitude Test (mean of 500 
and standard deviation of 100) . The standard scale was used in 
this research because of its interpretability . Scores on this 
exam, therefore, could range from 200 to 800. 

The test is reliable, both internally and over time 
(Jackson, 1984). The exam's validity was assessed by correlation 
with the Wechsler Adult Intelligence Scale and strong validity 
coefficients were obtained (Jackson, 1984). This test, although 
not quite as accurate a measure of intelligence as the Wechsler, 
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is, nevertheless, a good substitute measure of adult 
intelligence. 

Variables 

The variables used in this analysis consisted of ACT 
Assessment total score, ACT/COMP total score, MAB total score, 
high school grade point average (HSGPA) , and college grade point 
average (CUMGPA) , which was measured at the end of the sophomore 
year* MAB total score was an aptitude measure and the two grade 
point averages were achievement measures. 

The use of grade point average to measure achievement has a 
long history in educational research (Aldag and Rose, 1983; 
Savage, 1983; Bean and Kuh, 1984; Schuman, Walsh, Olson, and 
Etheridge, 1985) . It was felt in this research that HSGPA could 
identify the achievement component of the ACT Assessment and the 
ACT/COMP exam, while MAB total score would identify the 
intelligence component. 

Procedures 

Subjects for this study were recruited from their 
introductory psychology classes. They were offered two points of 
extra credit in their class and asked to sign up for a specific 
testing period. The MAB was administered to groups of 
approximately 20 individuals and the testing took about 1 hour 
and 15 minutes. The tests were hand scored, and results given to 
the test-takers. 
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After completion of the MAB, the student's ACT Assessment 
scores, freshman ACT/COMP scores, and high school GPA were 
retrieved from University records. Two years later or at the end 
of what would normally be their sophomore year, the student's 
cumulative GPA was also retrieved from the records. A few 
students had not completed two years of college when the GPA was 
retrieved and some had completed much more. In both these cases 
the GPA of record at that time was used. 

Data Analysis 

The data was analyzed using two different methods. First, a 
correlational analysis was performed to assess the relationships 
between the different variables. Then five regression analyses 
provided prediction models for the ACT Assessment, ACT/COMP, and 
cumulative GPA. The regression analysis used was Robust 
Regression advocated by Rousseeuw and Leroy (1987) . 

One problem with traditional regression analysis is that it 
is sensitive to outliers (extreme points in the y-direction) and 
leverage points (extreme data in the x-direction) . If an outlier 
or leverage point is present in the data, it can pull the 
regression line away from the center of the data and thus distort 
the descriptive model. Robust regression, rather than minimizing 
the least sum of squares, minimizes the least median of squares. 
This measure is less sensitive to outliers and leverage points 
and, therefore, produces a better descriptive model. 

Outliers identified through this method can be eliminated 
because, for whatever reason, they are extreme points that are 
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being affected by variables outside the model. The model that is 
left provides a better description and explanation of the vast 
majority of the data being analyzed. Thus, a more accurate model 
of most of the data is developed. 

RESULTS 

Descriptive Statistics 

Table 1 contains the descriptive statistics for the various 
measures used in this analysis. MAB scores are slightly above 



Table 1: Means and Standard Deviations of 
Test Scores 



Test 


Mean 


Standard 
Deviation 


Sample 
Size 


MAB 








Verbal 


555 


67.4 


133 


Performance 


560 


89.0 


133 


Full Scale 


559 


75.1 


133 


ACT Assess. 


21.5 


4.4 


120 


ACT/COMP 


173 


15.2 


110 


HSGPA 


3.22 


0.46 


126 


CUMGPA 


2.86 


0.60 


133 



the national average, while scores on the other measures were 
similar to those obtained by freshmen at UTK. The sample is 
representative of students at the University of Tennessee, 
Knoxville. 

It should be noted that there are different sample sizes for 
each measure. Although 133 students participated in the research 
and took the MAB, high school GPAs were only available for 126 of 
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them. Similarly, ACT Assessment scorer> were available for 120 
and ACT/COMP scores were available for 110 of them. The 
correlations and regression, therefore, use different sample 
sizes depending on the availability of data. 

Correlational Analysis 

Table 2 gives the correlations between the various measures. 
The MAB is strongly correlated with both the ACT Assessment exam 
and the ACT/COMP exam and less strongly correlated with the two 
achievement measures. The ACT Assessment is strongly correlated 
with the ACT/COMP Exam and moderately correlated with both 
achievement measures. The ACT/COMP, however, is not correlated 
with high school GPA, but is weakly correlated with Cumulative 
GPA. 



Table 2: Correlations between the Measures 



Measure 


ACT 
Assess . 


H. S. GPA 


CUMGPA 


ACT/COMP 
Total 


MAB 

Verbal 
Performance 
Full Scale 


.80*** 
.55*** 
.75*** 


. 37*** 
. 29*** 
.35*** 


.36*** 

.14 

.26** 


.59*** 
. 37*** 

.52*** 


ACT Assess. 




.46*** 


. 38*** 


.70*** 


HSGPA 






.50*** 


.18 


CUMGPA 








.26** 



*** p < .001 ** p < .01 



It appears from this data that the ACT Assessment has a 
stronger relationship with intelligence than it does with 
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achievement. The ACT/COMP exam also has a strong relationship 
with intelligence and no relationship with high school 
achievement. Tha finding that the ACT/COMP has a relationship 
with later achievement in college further suggests its role as an 
ability measure. To sort out these relationships a series of 
regression analyses were performed. 

Regression Analysis 

In the first analysis, ACT Assessment score was regressed on 
MAB Total Score and high school GPA. The Robust Regression 
determined that there were three outliers in this data and they 
wez v eliminated. In each case, the model under predicted the 
subject's ACT Assessment score. The final model was significant, 
F 2,^l 5=2 H4.52, p > .001, and explained 67% of the variance in 
ACT Assessment scores. Of this variance, MAB Total Score 
explained 45.7%, high school GPA explained 5.6%, and the 
interaction between the two measures explained 16%. Table 3 
contains the coefficients for this model. 



Table 3: Coefficients for the First 
Regression 



Parameter Estimate 



t-Ratio 



Prob. 



Intercept 



-11.35 



4.98 



< .001 



MAB Total 



0.04 



12.47 



< .001 



HSGPA 



2.48 



4.35 



< .001 
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The second regression analysis regressed ACT/COMP total 
score on MAB total score and high school GPA. This analysis 
identified two outliers — both of which were under predicted by 
the model. The final model was significant, F 2/ ioo =26.67, p < 
.001 and explained 35% of the variance in ACT/COMP scores. MAB 
total score explained 32.3% of the variance and the interaction 
between the measures explained 2.5%. High school GPA was not a 
significant element in this model of ACT/COMP scores. Table 4 
contains the coefficients for this model. 



Table 4: Coefficients for the Second 
Regression 



Parameter 


Estimate 


t-Ratio 


Prob. 


Intercept 


108. 


55 


9 


.77 


< .001 


MAB Total 


0. 


12 


7 


.04 


< .001 


HSGPA 


-0. 


51 


0 


.18 


.8565 



As MAB total score had explained so much of the variance in 
ACT Assessment score, a third regression analysis was performed 
regressing ACT/COMP score on ACT Assessment Score and high school 
GPA. Two outliers were also identified in this analysis — one 
over predicted ACT/COMP score and one under predicted it. The 
model from this analysis was also significant, F 2/ 93 = 48.82, p < 
.001, and explained 51% of the variance in ACT/COMP scores. As 
with the previous model, high school GPA was not a significant 
predictor. ACT Assessment, however, explained 47.4% of the 
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variance and the interaction between the predictors explained 
2.2%. The coefficients for this model are in table 5. 

To assess the predictability of college GPA from MAB scores 
or ACT Assessment scores, two final regressions were performed. 
The first analysis regressed Cumulative GPA on MAB scores and 



Table 5: Coefficients 
Regression 


for the 


Third 


Parameter 


Estimate 


t-Ratio 


Prob. 


Intercept 


133.23 


16.25 


< .001 


ACT Asses 


2.55 


9.50 


< .001 


HSGPA 


-4.62 


1.74 


0.0857 



high school GPA, while the second substituted ACT Assessment 
scores for MAB Scores. In the first analysis, three outliers 
were identified and in each case cumulative GPA was over 
predicted. 

In the final model for this analysis, the intercept term was 
not significant and the analysis was rerun through the origin. 
This final model had an F 2/ i21 = 2521.24, p < .001 and explained 
98% of the variance in cumulative GPA with the vast majority of 
the explained variance assigned to the interaction of MAB total 
score and high school GPA. Table 6 contains the coefficients for 
this model. 
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Table 6: Coefficients for the Fourth 
Regression 



Parameter 


Estimate 


t-Ratio 


Prob. 


MAB Total 


0. 


001 


2. 


13 


0.0354 


HSGPA 


0. 


73 


8. 


84 


< .001 



The analysis regressing cumulative GPA on ACT Assessment and 
high school GPA identified four outliers that over predicted 
cumulative GPA. Also, this model, as the previous model, had an 
insignificant intercept and the regression was rerun through the 
origin. This final model had an F2 / H4 = 1776.67, p < .001 and 
explained 97% of the variance in cumulative GPA with most of the 
explained variance assigned to the interaction of ACT Assessment 
and high school GPA. Table 7 contains the coefficients for this 
model. Both these models were highly significant with ACT 
Assessment scores and MAB scores similarly interacting with high 
school GPA to explain almost all the variance in cumulative GPA. 



Table 7: Coefficients for the Fifth 
Regression 



Parameter 


Estimate 


t-Ratio 


Prob. 


ACT Asses 


0. 


032 


2. 


68 


0.0084 


HSGPA 


0. 


67 


8. 


29 


< .001 



DISCUSSION 

Correlation analysis and regression analysis were performed 
to assess the relationship between both the ACT Assessment exam 
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and the ACT/COMP exaiu and intelligence. The correlation analysis 
showed that both exams had a stronger relationship with 
intelligence than they did with achievement as measured by grade 
point average. In fact, the ACT/COMP exam had no relationship 
with high school GPA at all. The regression analyses showed that 
intelligence was a strong predictor of ACT Assessment scores and 
achievement, while significant, did not explain much of the 
variance in scores. Models of the ACT/COMP exam results using 
both the MAB and the ACT Assessment for predictors explained 
significant variance in those scores but the models did not 
include high school GPA. Finally, models for college GPA 
explained 98% of the variance, but almost all this explained 
variance was due to the interaction of the ability measure and 
high school GPA. 

This data suggests that both the ACT Assessment and the 
ACT/COMP exam are measuring ability rather than achievement. The 

r 

correlations between these two scores and intelligence are high 
and regression analysis shows that intelligence is a much 
stronger predictor of these test scores than the achievement 
measure. In addition when intelligence test scores and ACT 
Assessment test scores are used in a prediction of college GPA, 
they explain similar amounts of variance. 

This is the first study to examine the relationship between 
intelligence and ACT Assessment scores and ACT/COMP scores. 
Therefore, future replications would be in order. While there is 
no literature to compare this study too, this research is 
supportive of actual practice in the field. Many studies have 
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used the ACT Assessment exam as an ability measure (Snowman, 
Leitner, Snyder, and Lockhart, 1980; Ashmore and Cork, 1985; 
Pedrini, Tritchler, and Pedrini, 1984; Pike and Banta, 1989), and 
this study confirms that decision • 

Two cautions are in order at this point • First, this 
research used the ACT Assessment exam and not the Enhanced ACT 
currently available. While this research probably would 
generalize to the Enhanced ACT, further research is necessary to 
confirm that the relationship with intelligence exists there 
also. 

Second, it is hard to remove the effects of achievement from 
any paper and pencil intelligence test. This research sought to 
do so by using the total score from the MAB, which included a 
performance subscale, rather than the subscale scores. However, 
some variance assigned to ability may actually be the result of 
high school achievement. It is doubtful, however, that a better 
measure of intelligence or achievement would substantially change 
the conclusions of this study. 

It appears that the ACT Assessment can be substituted for a 
more formal test of intelligence with little loss of predictive 
power. It also appears that the ACT/COMP exam is measuring 
little other than ability when it is used for assessment 
purposes. While further research is needed to confirm these 
findings, they certainly call the use of the ACT/COMP exam for 
college outcomes assessment into question. 
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